AI based image analysis of red blood cells in oscillating microchannels

The flow dynamics of red blood cells in vivo in blood capillaries and in vitro in microfluidic channels is complex. Cells can obtain different shapes such as discoid, parachute, slipper-like shapes and various intermediate states depending on flow conditions and their viscoelastic properties. We use artificial intelligence based analysis of red blood cells (RBCs) in an oscillating microchannel to distinguish healthy red blood cells from red blood cells treated with formaldehyde to chemically modify their viscoelastic behavior. We used TensorFlow to train and validate a deep learning model and achieved a testing accuracy of over 97%. This method is a first step to a non-invasive, label-free characterization of diseased red blood cells and will be useful for diagnostic purposes in haematology labs. This method provides quantitative data on the number of affected cells based on single cell classification.


Introduction
Blood tests still play a major role in diagnostics since samples are easy to obtain and are routinely used for analysis in clinical and general practice settings. 1Red blood cells (erythrocytes) form by far the major component of whole blood (about 50%) and have been used as biological indicators for several diseases such as sickle cell anaemia, spherocytosis, beta-thalassemia, malaria, 2,3 hypercholesterolemia 4,5 and many others.In many of these diseases the shape of the erythrocyte is altered, so simple optical microscopical examination and cell counting allows quantitative analysis in stasis e.g., in a blood smear.The adopted shape depends on mechanical properties of the cell and the external strain.In hydrodynamic ow, there have been various shapes reported in vivo as well as in vitro.For healthy cells these encompass discocytes, parachutes and slipper-like shapes and echinocytes, spherocytes, etc. for diseased cells.However, not all cells of a sample are in the same shape condition, due to variances in mechanical properties within the cell population.Even within one blood sample there exists a variance of mechanical properties of the erythrocyte since for example cells alter their properties and size during their ∼120 days lifetime of circulation in the organism. 6Red blood cells (RBCs) in microow conditions have been analysed both in experiments 7 and theory. 8Most of the studies are in capillary ow, yet some research was done on more complex geometries to reveal the shape relaxation dynamics. 9,10sed on viscoelastic models the experimentally observed cells could be reproduced and variation of mechanical parameters (viscoelastic moduli) were determined has also demonstrated that transitions in between these cell shapes can occur.However, in experimental settings it is still challenging to distinguish between populations of red blood cells from different samples, such as diseased cells and healthy cells.
In AI based red blood cell classication most of the prior work [11][12][13] distinguished between populations of cells with the associated problems mentioned above and in addition poses technical problems when employing segmentation techniques to isolate individual blood cells which are error prone and computationally intensive.Because there is a strong interest in the viscoelastic properties of the red blood cells oen only their shape has been analysed but not the healthiness of the cells. 14,15he use of a channel to probe the mechanical properties of the red blood cells in conjunction with AI to detect a disease has, to our knowledge, only attempted once. 168][19][20][21][22][23][24] Unlike the other studies mentioned before, we do not analyse the contour or shape of the cell to model the cell mechanics but train a TensorFlow implemented AI model with a large number of images from native and treated cells.The model is trained for 10 epochs and achieves a high training and validation accuracy already aer ∼5 epochs, making is very efficient.Experimentally, we use a microchannel with oscillating width to transiently deform cells to capture both, the elastic, and the viscous properties.Our device presents a method that can distinguish a population of cells which are not obvious by simple inspection by eye.It is ready to be used for disease diagnosis and analysis of the severity of a disease by providing quantitative results on the frequency of affected cells and enables an end-toend classication of red blood cells without understanding the biomechanical differences between the classied cells in detail.

Microuidic procedures/setup
We probe red blood cells in ow with periodically oscillating ow velocities in a zigzag-shaped microchannel.The channels are fabricated using standard so-lithography and mounted on an inverted uorescence microscope.Videos of cells owing through the channel are recorded with a high-speed camera (Photron, UX50).Red blood cell samples are prepared from whole blood by centrifugation and incubation steps with chemicals for modication of their mechanical properties (see material and methods section).We use a diluted red blood cell suspension in PBS buffer with a haematocrit (Ht) of Ht = 0.5% and inject it into the inlet of the microuidics device using a pressure driven system, controlling the pressure drop between inlet and outlet.Cells then enter a zigzag-shaped region that is in the eld of view of a 60× objective as shown in Fig. 1.To avoid large sections of videos without cells and to reduce memory consumption at the low chosen, we trigger the recording of videos by a hardware trigger.Therefore, we use a laser to excite (488 nm) a uorescent marker in the red blood cell and detect the emitted (525 nm) uorescence signal with a photomultiplier tube (PMT).In this way we record cell sequences of 60 and 40 frames per cell.We record 248 sequences and then transfer the batch of data from the fast camera RAM to the computer hard drive for post-processing.

Video processing
Three batches of 248 sequences each were recorded for healthy cells which results in a total of 744 video sequences of native red blood cells (the rst two sequences where removed).Two batches of 248 sequences each were recorded for chemically modied cells which resulted in a total of 496 video sequences.Fig. 2 shows how the training data has been extracted from the raw video sequences.Fig. 2A shows a snapshot of one clip where the cells are owing from the le to the right.Here, the cell has already progressed to the 2nd narrow section of the channel.Since we are interested how the zigzag-channel impacts on the cell's behaviour and shape, we have dened a "so trigger" where a snapshot for training is used.One can think of the so trigger as a nishing line in a race.
As an example, we have shown one so trigger when the cell in the red square has just arrived at a narrow section and has touched the so trigger.A 2nd so trigger has also been established (not shown) when the cell has arrived at a wide section and thus, we have two so triggers: one for taking a snapshot in the narrow section and one for the section.Once the cell has been detected a snapshot is taken of the size of the crop window indicated with "crop" in Fig. 2A.As a next step the background was obtained (Fig. 2B) and subtracted (Fig. 2C) to prevent TensorFlow from learning features of the background instead of that of the red blood cells.Since the cell ows through the image from le to right, we can take the background for the le half from the nal frames of a clip and the background from the right half from the rst frames of the clip.Splicing these two halves together gives the background which can then be used in Fig. 2C to obtain a background-free cell image.

Data analysis and decision making with TensorFlow
The video processing pipeline described above uses three batches of 248 images (one image per sequence) of native red blood cells and two batches of 248 images of chemically modied red blood cells for training.For training the model every image is labelled of either being healthy or diseased.This results in 5x248 images/label-pairs.These pairs are then  shuffled randomly for training and then fed into TensorFlow (Fig. 2D).This is a standard network topology as suggested by TensorFlow/Keras to do image classication.The network has two outputs, one gives the probability of being a native cell and one the probability of being a chemically modied one.Training was performed with 10 epochs and the test accuracy evaluated with two separate batches of clips not being part of the training: 492 images of healthy red blood cells and 111 images of diseased cells.These two subpopulations where then sent into TensorFlow for training (80%) and validation (20%) which is described next.

Comparison of frame location in the microuidic channel
To investigate the impact of the position of the cell in the microchannel we created two datasets of red blood cell images and fed them into TensorFlowone with the red blood cells detected in the widening section of the microchannel and another one with the red blood cells in the narrow section as shown in Fig. 3.The goal of training is to have for a two-output classier one output close to one and its other output close to zero so that either the "native" output becomes one or the "chemically modied" becomes one but never both.TensorFlow training and validation on the dataset containing images from the narrow part of the channel is shown in Fig. 3A: on the le the training and validation accuracy is shown and on the right the training and validation loss is shown.Both training and validation accuracy reach 99% aer 5 epochs and stabilise there.The same applies for the training and validation loss which reach their minimum aer 5 epochs.Training using images from the wide section (Fig. 3B) of the microchannel result is remarkably similar behaviour: both the training accuracy and the validation accuracy converge to over 99% aer 5 epochs.Also, the training loss and validation loss reach their minima in a similar way as the training in the narrow section.
Aer training the TensorFlow model was presented with test data which the network has not yet seen: 492 clips of native red blood cells and 111 clips of chemically modied cells.The testing accuracy at the narrow section was 98% and for the wide one 97%.Fig. 3C/D show the testing results for the dataset generated at the narrow section of the channel and Fig. 3E/F at the wide section of the channel.Every row shows three examples of red blood cells and their detection probability against the ground truth: true native or true chemically modied.Remember that training forces the network to clearly output a one for its detection category.For example, in Fig. 3C the ground truth is "native" and the network outputs a one for native, indicated as "N" under the green bar.Then in Fig. 3D the cells were chemically modied, and the network classied them all with very high condence as chemically modied ("C").Fig. 3E/F now shows the same as above but for the wide section of the channel.Again, true native ones are detected as native without any doubt and reects the 97% of the testing accuracy.The truly chemically modied ones were also detected with ease.Having just two categories being native or chemically modied allows a simple comparison which is robust with one category being almost zero and the other almost one.

Microuidic device preparation
Microuidic channels are fabricated using so lithography. 25,26riey, the channel structure is designed in CAD program and transferred to a chromium mask (ML&C GmbH).The zigzagshaped channel has a periodicity of 20 mm, and an amplitude of 20 mm.The narrow part of the channel is 10 mm and the wide part is 20 mm in width (for details see sketch in the ESI †).To structure of the mask is transferred to a silicon wafer coated with 10 mm SU8-3010 photoresist that denes the height of the channel (Microchem, SU8 3000 series) using a mask aligner (MA6, Süss MicroTec).Aer developing with (Microposit™EC Solvent, Shipley), the structured SU8 acts as a template to fabricate PDMS moulds (PDMS, Sylgard™ 184 Silicone Elastomer Kit).The uid PDMS is poured onto the template and cured for 4 h in an oven at 75 °C.The ratio of elastomer base to curing agent is 10 : 1.
We punch holed into the cured PDMS mould to connect inlet and outlet to the tubing.The PDMS is covalently bonded onto a microscope slide using oxygen plasma.

RBC preparation
Whole blood was obtained from healthy donors (Research Donors, Cambridge Bioscience) and washed three times in a phosphate-buffered saline solution (PBS 1×, pH 7.4, 307 mOsm L −1 , Gibco Life Technologies).Aer each washing step, the sample was centrifuged for 5 minutes at 2500 rpm (mini spin plus centrifuge, Eppendorf), and the white buffy coat and supernatant were removed.
For the native red blood cell sample, we used 10 mL of concentrated RBC pellet from the bottom of the reaction tube and incubated in 0.5 mL green-uorescent calcein-AM solution (5 mM, Invitrogen) for 30 minutes at 37 °C, followed by three washes with PBS.
For the chemically modied RBC experiments, we rst incubate the red blood cells in formaldehyde solution before we stain the cells.Briey, 10 mL of the pellet were incubated in 0.37% formaldehyde solution (Sigma-Aldrich) for 10 min at room temperature.The cell suspension was then washed three times before stained in calcein-AM as described above.
To avoid cell sedimentation during the experiment, the cells were suspended in a density-matched solution using a density gradient solution (OptiPrep Density Gradient Medium, Sigma Life Science).Optiprep is a sterile non-ionic solution of 60% (w/ v) iodixanol in water.Additionally, bovine serum albumin (BSA, Ameresco) was used to prevent cell adhesion to each other and the microchannel walls.The stock density matched solution was prepared by mixing 945 mL of OptiPrep solution with 3035 mL of PBS and 40 mg BSA.
For the experiments 5 mL of the native or chemically modi-ed cell suspension were mixed with 995 mL of the stock density matched solution and used as the stock solution with a haematocrit of 0.5%.The experiments were completed within a day of blood collection.The nal density of this solution was r = 1.080 g mL −1 with an osmolarity of c osm = 317 mOsm L −1 .The viscosity of this solution was determined aer Ubbelohde to be h = 1.5 ± 0.1 mPa s.
The red blood cell suspension was pumped by a pressure driven ow using a pressure transducer (MarshBellofram, USA) and a pressure of p = 20 mbar at cell velocities in range from 1.3-8 mm s −1 .Due to the Poiseuille-like ow cells owing close to the wall are slower than cells in the centre of the channel.The average velocity of the red blood cells measured is v RBC =5.1 ± 2.0 mm s −1 .The ow velocity is not constant and does change along the channel because of the alternating width and across the channel due to the Poiseuille-prole.Similarly, the ow rate is variable.However, to estimate typical shear forces on the cell we estimate an effective shear rate from an average channel width of w av = 15 mm and the average velocity to be _ g eff = 2v RBC w av = 680 s −1 .From the shear rate and the viscosity of the density matched solution h = 1.5 mPa s we estimate an effective shear stress to be s eff = h × _ g=1.0 Pa.

Python code
The red blood cells videos consist of multiple sequences conjugated aer each other triggered by the uorescence detector described in Fig. 1.The number of sequences in each video can vary in the number of frames and is specied in the code for the different videos used.We use two "nishing lines" at different positions in the oscillating channel where we wanted to capture images of cells at the wide and narrow sections.The nishing lines were different for wide and narrow section at each of the videos and it was inputted in the code specic for the different videos as before.
To detect the cells at these nishing lines, we used OpenCV background subtractor, a Gaussian Mixture-based background/ foreground segmentation, with a history of 100 and varThreshold of 20.We exclusively applied this only to the specic detection area which was determined by the nishing line parameter we had specied.The detection area was dened by a y range of (10 110) and an x range from the nishing line to the nishing line plus 250.Then, we used OpenCV function "ndcontours" to create contours around the moving cells in that detection area.We dened an area threshold of 20 px 2 enclosed by these contours to identify a cell.If a cell passes the detection area and is above the set threshold the frame number is recorded and an image of the cell is extracted.In this way the function "ndcontours" is only used to determine the sotrigger as shown in Fig. 2 and no information on the contour is used in the AI categorization model.
The resulting images were cropped to a size of 120 × 150 pixels, with a cropping range of 0 to 120 for height and from the nishing line minus 80 to the nishing line plus 70 for the width.Therefore, images were of a shape of (120,150,3).
For our neural network, we used Keras sequential model with 10 layers.The rst layer is used to rescale and normalize the input data.In the second layer a 2D convolution layer is used with 16 lters to detect different features in the input image.The next layer, the Maxpooling2D layer, reduces the spatial dimensions of the data, controlling overtting and reduces the computational load.In the fourth layer another 2D convolutional layer is used but with more lters as before, with 64 as the number of output lters.Then another Maxpooling2D layer to reduce the dimensions of the data is used.To prepare the data for the upcoming dense layers the multidimensional output from the previous layers is attened into a onedimensional vector.For the two dense layers we used rst 128 units per neuron to learn the complex relationship between the attened features followed by the last layer, another dense layer with two units representing the output classes: "native" and "chemically modied".To extract all pixel information from the images we used in the 2D convolutional layers a stride parameter of (1,1) and a kernel size of (3,3).Finally for the training, we passed through the entire training dataset ten times (10 epochs) in batch sizes of 32 samples where aer each batch the gradients were updated.A schematic representation of the network is shown in Fig. 2D.

Discussion
In this study we have used periodically oscillating zigzag-shaped microchannels to probe RBCs.Compared to many other studies the spatially alternate design enables the interrogation of viscous cell properties.In contrast, RBCs in cylindrical capillary ow adopt a constant shape and neither dynamic, viscous properties of the RBC cell membrane nor the cytosol enter because the cell moves as a xed object in ow.Here, we probed cells at two separate locations in microow, at the widening and the constriction.In between these extrema, the RBCs adopt an intermediate shape.The transition between these shapes in the wide and narrow section is controlled by the relaxation time of the RBC as given by the ratio of viscous to elastic parameters.Here, we tested native and chemically modied RBCs using formalin at 0.3% volume ratio (formaldehyde in water solution).Formalin is a xation agent and non-specically cross-links proteins in the RBC thereby modifying the viscoelastic, mechanical properties of the cell.Similar to glutaraldehyde (GA), the aldehyde group binds and can interlink amino acids of proteins and thereby mechanically stabilize the RBC membrane.In GA mediated mechanical modication, the GA can exist as a monomeric or polymeric form, both of which bind to amino acids.Therefore, GA can link proteins over a variable distance depending on the length of the polymer.Even though formalin can also form formaldehyde polymers, the mechanically reactive form with an aldehyde group is the monomer, i.e. methylene hydrate.Only as monomer it can bind to proteins to form a methylene cross-link.Therefore, the potential for crosslinking with GA is much larger, rstly because of its two aldehyde binding sites and secondly due to the variable lengths. 23ence, the effect of formalin is milder and slower than the one of GA, facilitating chemical control of mechanical RBC properties.However, due to the small size formalin has been reported to have a much higher permeation rate compared to GA. 27 It can potentially enter the RBC and link proteins in the cytosol, such as hemoglobin. 27n AFM studies an increase in elastic modulus depending on GA concentration has been reported as well as a reduction in deformability (elongation index). 23AFM measurements using formalin at a concentration of 5% have revealed a 10-fold increase in Young modulus as compared to untreated cells. 28he difference in stiffness leads to less deformation and a lower elongation index as obtained from a shape analysis. 29tiffening of RBC membrane has also been achieved by incubation in diamide. 24,30,31However, diamide has been reported to only rigidify the membrane stiffness and uidity.It provides disulphide bridges between specic thiol-group containing amino-acids (cystine, methionine) and has a minor effect on the cytosol.Its effect has been studied in microow analysing the shape changes by using the cell width to length ratio. 32n our experiments, the changes in viscoelastic properties due to chemical treatment with formalin 0.3% are expected to affect the morphology of the RBC in ow.However, we could not detect any apparent shape changes in our micrographs by eye or simple analysis such as analysing the deformation index or projected area.
Instead of analysing the RBC shape and contour as we have done in previous studies, 33,34 here we have taken an AI based approach. 35 few other authors have used AI based analysis in micro-uidics channels.Lamoureux et al. 36 used a microuidic ratchet channel to sort highly deformable RBCs and trained the machine learning on the isolated cells.Darrin et al. 44 have used millimetre sized channels and observed red blood cells near the wall at shear rates of 10 s −1 to study the motion from tumbling to tank treading.The channel size used by Recktenwald et al. 37 and Kihm 14 et al. have similar dimensions as our geometry and both are operating in the same velocity range of up to 10 mm s −1 .However, they use a straight channel and different ow rates to investigate the shape transition of individual red blood cells and therefore their device is mainly sensitive to elastic parameters.
Alkrimi et al. 15 have classied RBCs using machine learning principal component analysis to reduce the correlation of features.Yet, the morphological analysis was done on a blood smear and not involving microuidic channels.
Using a TensorFlow based machine learning we categorize native and chemically treated cells with high precision and with less training data required because we use a pretrained model.
This goes beyond current literature that probe red blood cells in simple capillary ow. 16e examined how the position of cells within an oscillating microchannel affects the outcome of the categorization.Two sets of red blood cell images were used, one in the channel's wider section and the other in the narrower section.Using TensorFlow, an image classier with two outputs was trained and its objective was to generate one output close to one while maintaining the other output close to zero, thus, effectively distinguishing between "native" and "chemically modied" cells.The training accuracy for images of both section in the microchannel exceeded 99% aer only ve epochs, demonstrating the fast and efficient training convergence of the model.Similarly, the validation accuracy converges rapidly as shown in Fig. 3A and B. As a second indicator we used the training and validation loss, a parameter that quanties how bad the prediction of the model is (penalty for poor prediction).Both indicators rapidly converge to small value indicating a very good prediction of the model.In both, the narrow and wide part of the channel, the validation and training accuracies and loss are monotonous increasing and decreasing, respectively and the uctuation of their values with epochs is small.The value of all indicators plateau within about 5 epochs, again showing a very efficient categorization.
The trained model was then tested with previously unseen data of native and chemically modied cells and yielded a 98% accuracy rate for the narrow section and 97% for the wide section.Apparently, the binary categorization of cells does only weakly depend on the position of the cell and suggests that the selection of the position is not critical to the prediction of the image-based AI model.
Central to red blood cell diagnostics is the analysis of their deformability and, thus, their ability to form different shapes. 16,36For example, Kihm et al. divided the cells in two groups called "Slippers" and "Croissants", 14 implemented a CNN (Convolutional Neural Network) to train 4000 images and classify the RBCs uniquely based on their shape characteristics.Considering the increasing demand for advancements and the potential for signicant impact and popularity in this eld, Recktenwald et al. 37 and its follow-up study 38 adopted the approach, proposed by Kihm et al. and Alkrimi et al., 14,15 to benchmark different AI techniques classifying RBCs and similarly to Kihm et al., 14 the classication was based fully on morphology.Lee et al. 39 uses not just shape but also texture features to classify normal and abnormal RBCs and similarly to Das et al., 12 they classify cells in more than two categories.However, overall, these studies employ an intermediate step by rst or exclusively focussing on morphology and then feeding the pre-processed data into the nal classier for diagnosis.However, this assumes that one knows which deformation, shape or texture relates to a diseased cell and which one to a healthy one.In contrast, we do pixel-to-disease-classication (i.e., end to end) where the deep net learns the distinguishing features by itself without 1st hand-craing features and only then feeding them into a classier.
Oen raw microscopic images taken contain a large amount of red blood cells which requires a segmentation process to extract individual images. 16,39Due to the segmentation processes and individual image extraction, the quality of the image is oen seen reduced and as a consequence degrades the classication accuracy.Other approaches to work with images of many red blood cells are the Circular Hough transform 40 or region-based segmentation (ORBS). 13Das et al. 12 also studies segmentation of RBCs in image classication and similarly to the other studies 40 and the study of Shemona et al., 13 the images analysed also contain many RBCs.In contrast to these studies our work directly captures single red blood cells passing through the microchannel and is not affected by image degradation, low resolution aer cropping or wrong segmentation.
AI can be used to classify RBCs into more than two categories, 12,39 for example Malaria, Thalassaemia, other abnormal and normal. 11In our study we have used two categories as a proof of concept but the TensorFlow classier can take any number of classes where the only limit is the available computing power and GPU memory.Future work will use more than two categories.
Confounding factors are a signicant challenge in any AI based learning algorithm which is at risk of learning just trivial and supercial features from the training data.For instance, Zech et al. explains that CNNs may not effectively identify disease-specic nings but rather exploit confounding information. 41This crucial issue is only addressed by Rizzuto et al. which tries to eliminate any confounding structures cropping the video to extract only a limited area of interest. 16No study has explored the effect of confounding factors such as focal plane on image classication of RBCs.In this study we discovered that AI would use the focal plane to distinguish between native and chemically modied RBCs when using one focal plane for the native ones and one for the diseased ones.To overcome this problem, we mixed images captured at different focal planes and, in addition, removed the background from the images.

Conclusion
We have demonstrated an end-to-end classication of two populations of red blood cells with an open-source AI image analyser based on TensorFlow.Cells of both populations ow through a microchannel and images are taken at specic locations in the channel.The model very effectively classies the cell into native and chemically modied and reaches a high level of accuracy already aer 5 epochs.The fully trained model was tested with an unknow sample, and the testing accuracy was 98% and 97% in the narrow and wide section of the channel, respectively.The high level of accuracy and the quickly converting low level of testing and validation loss justies our approach.In the future we aim to extend the analysis to other blood samples from patients for disease diagnostics and combine the analysis with droplet based microuidics and red blood cell encapsulation 42 as well as acoustic approaches for viscoelastic cell classication. 43

Fig. 1
Fig. 1 Schematic of the microfluidic setup mounted on an inverted fluorescence microscope: RBCs entering the device and flow into the region of oscillating width and adopt their shape.Videos are taken with a fast camera and recording is triggered by cell passing a photoelectric barrier and rapid analysis in a photomultiplier tube.

Fig. 2 2
Fig. 2 2 Data processing pipeline.(A) Single frame from a video clip with a red blood cell in the red boundary box touches the "soft trigger" (black line) which then leads to the extraction of the image in the area "crop" (black box).(B) Background of the cropped area from (A) by taking a video frame from the video clip without a red blood cell.(C) Background subtraction of (B) subtracted from (C) and taken the absolute value.(D) TensorFlow layers: "norm": RGB value normalisation to 0.01, "conv2D": standard 2D convolutional layer, "max pooling": standard 2D max pooling, "dense": standard dense layer.The final layer has two outputs: one for the detection probability of a native red blood cell and one for the chemically modified one.

Fig. 3
Fig. 3 TensorFlow training behaviour and results.(A) Training and validation loss for the narrow part of the microchannel and for the wide part (B) of the microchannel.Classification results for the narrow section of the microchannel for (C) native red blood cells and (D) chemically modified red blood cells.Classification results for the wide section of the microchannel for (E) native red blood cells and (F) chemically modified red blood cells."N": native, "C": chemically modified, "p/det": detection probability.